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Abstract. A new variant of the Compressed Sensing problem is inves¬ 
tigated when the number of measurements corrupted by errors is upper 
bounded by some value l but there are no more restrictions on errors. We 
prove that in this case it is enough to make 2 (t + l) measurements, where 
t is the sparsity of original data. Moreover for this case a rather simple 
recovery algorithm is proposed. An analog of the Singleton bound from 
coding theory is derived what proves optimality of the corresponding 
measurement matrices. 


1 Introduction and Definitions 

A vector x = {x \,..., x n ) £ R n in n-dimensional vector space R" called t- 
sparse if its Hamming weight wt(x) or equivalently its Iq norm ||a;||o is at most 
t, where by the definition wt(x) = ||x||o = |{i : Xi y 0}|. Let us recall that 
the Compressed Sensing (CS) Problem (1121 is a problem of reconstructing of an 
n-dimensional t-sparse vector a: by a few (r) linear measurements = (h^ l \x) 
(i.e. inner product of vectors x and h W), assuming that measurements (h^\x) 
are known with some errors e», for i = 1,... ,r. Saying in other words, one needs 
to construct an r xn matrix H with minimal number of rows h^ l \ ..., h^ r \ such 
that the following equation 

s = Hx t + e, ( 1 ) 

has either a unique f-sparse solution or all such solutions are “almost equal”. 
The compressed sensing problem was mainly investigated under the assumption 
that the vector e = (ei,..., e r ), is called the error vector, has relatively small 
Euclidean norm (length) ||e|I 2 - We consider another problem’s statement assum¬ 
ing that the error vector e is also sparse but its Euclidean norm can be arbitrary 
large. In other words, we consider the doubly sparse CS problem when ||a;||o < t 
and ||e||o < l. The assumption ||e||o < l was first time considered in [3] as a 
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proper replacement for discrete version of CS-problem of usual assumption that 
an error vector e has relatively small Euclidean norm. 

Definition 1. A real r x n matrix H called a ( t,l)-compressed sensing (CS) 
matrix if 

\\Hx T -Hy T \\ 0 > 21 + 1 (2) 

for any two distinct vectors x,y £ R n such that ||.t||o < t and ||y||o < t. 

This definition immediately leads (see ®) to the following 
Proposition 1. A real r x n matrix H is a (t, l)-CS matrix iff 

||tfz T ||o>2Z + l (3) 

for any nonzero vector z £ R" such that || 2 ||o < 2 1. 

Our main result is an explicit and simple construction of (t,l )-CS matrices with 
r = 2 (t + l ) for any n. We show this value of r is the minimal possible for (t, l)- 
CS matrices by proving an analog of the well-known in coding theory Singleton 
bound for the compressed sensing problem. Besides that we propose an efficient 
recovery (decoding) algorithm for the considered double sparse CS-problem. 

2 Optimal Matrices for Doubly Sparse Compressed 
Sensing Problem 

We start with constructing of (t, Z)-CS matrices. Let a real f x n matrix H be a 
parity-check matrix of an (n,n — f )-code code over R, correcting t errors, i.e. any 
2 1 columns hi ,,..., hi 2t of H are linearly independent. And let G be a generator 
matrix of an (r, f)-code over R of length r, correcting l errors. Let matrix H 
consists of the columns hi ,..., h n , where 

hj = hjG (4) 

and transposition T means, that vectors hj and hj are considered in (4) as row 
vectors, i.e. 

H = G t H (5) 

In other words, we encode columns of parity-check matrix H , which is already 

capable to correct t errors, by a code, correcting l errors, in order to restore 
correctly the syndrom of H. 

Theorem 1. Matrix H = G T H is a ( t,l)-CS matrix. 

Proof. According to Proposition 2 it is enough to prove that ||fL3 T ||o >21 + 1 
for any nonzero vector 2 £ R" such that ||^||o < 2 1. Indeed, u = Hz T ^ 0 since 
any 2 1 columns of H are linear independent. Then Hz T = G T Hz T = G T u = 
(u T G) T and u T G is a nonzero vector of a code over R, correcting l errors. Hence 
||F2 t ||o = ||w t G||o>2/ + 1. □ 


Now let us choose the well known Reed-Solomon (RS) codes (which are a 
particular case of evaluation codes construction) as both constituent codes. The 
length of the RS-code is restricted by the number of elements in the held so in the 
case of R the length of evaluation code can be arbitrary large. Indeed, consider 
the corresponding evaluation code RS( n ^) = {(/(ai,..., f{a n )) : deg f(x) < 
k }, where ai,...,a n € R are n different real numbers. The distance of the 
code d = n — k + 1 since the number of roots of a polynomial cannot 
exceed its degree and hence d > n — k + l,but, on the other hand, the Singleton 
bound states that d < n — k + 1 for any code, see g|. Therefore the resulting 
matrix if is a (t,l)-CS matrix with r = 2 (t + l ). The next result, which is a 
generalization of the Singleton bound for the doubly sparse CS problem, shows 
these matrices are optimal in the sense having the minimal possible number r 
of linear measurements. 

Theorem 2. For any ( t,l)-CS r x n-matrix 

r > 2(t + l). (6) 

Proof. Let H be any (t , /)-CS matrix of size r x n, i.e., ||.HT t ||o > 21 4- 1 for any 
nonzero vector z € R" such that 11^||o < 2 1. And let H 2 t-i be the (2 1 — 1) x n 
matrix consisting of the first — 1 rows of H. There exists a nonzero vector z = 
(zi ,..., Z 2 t, 0,0,..., 0) eR" such that Hz T = 0 (a system of linear homogenious 
equations with the number of unknown variables larger than the number of 
equations has a nontrivial solution). Then ||7Lz T ||o < r — (2i — 1) and finally 
r > 2t + 21 since ||IL£ T ||o > 21 + 1. □ 

3 Recovery Algorithm for Doubly Sparse Compressed 
Sensing Problem 

Let us start from a simple remark that for e = 0 recovering of the original sparse 
vector x , i.e., solving the equation (1), is the same as syndrome decoding of 
some code (over R) defined by matrix H as a parity-check matrix. In general, 
syndrome s = Hx T is known with some error, namely, as s = s + e and therefore 
we additionally encoded columns of H by some error-correcting code in order to 
recover the original syndrome s and then apply usual syndrome decoding algo¬ 
rithm. Therefore recovering, i.e., decoding algorithm for constructed in previous 
chapter optimal matrices is in some sense a “concatenation” of decoding algo¬ 
rithms of constituent codes. 

Namely, first we decode vector s = s + e by a decoding algorithm of the 
code with generator matrix G. Since ||e||o < l this algorithm outputs the correct 
syndrome s. After that we form the syndrome s by selecting first f coordinates 
of s and then apply syndrome decoding algorithm (of the first code with parity- 
check matrix H ) for the following syndrom equation 


s = Hx T . 


( 7 ) 


Now let us discuss a right choice of constituent codes. It is very convenient to 
use the class of Reed-Solomon codes over R. There are well known algorithms of 
their decoding up to half of the code distance (bounded distance decoding, see 
bd> for instance, Berlekamp-Massey algorithm, which in our case (codes over R) 
is known also as Trench algorithm, see 0. Hence the total decoding complexity 
does not exceed 0{rr) operations over real numbers. Moreover we can even de¬ 
code these codes over their half distances by application of Guruswami-Sudan 
list decoding algorithm [5]. 

It is well known that encoding-decoding procedures of Reed-Solomon codes 
become more simple in the case of cyclic codes, when the set a±,... ,a n is a 
cyclic group under multiplication. In order to do it let us consider ai,...,a n 
as complex roots of degree n and define our codes through their “roots”, i.e. 
our codes consist of polynomials f(x) over R such that ) = 0 for m € 

{— s, ..., — 1 , 0, +1,..., +s} with s = t for the first constituent code and s = l 
for the second. It easy to check that such codes achieve the Singleton bound 
with d = 2s + 2 , so the corresponding doubly sparse code has redundancy r = 
2(t + l + 1) what is slightly larger than the corresponding Singleton bound, but 
in return these codes can be decoded via FFT. 

4 Discussion - no small errors case and slightly beyond 

Let us note that the initial papers on Compressed Sensing especially stated that 
this new technique (application of 1 1 minimization instead of Iq) allows to recover 
information vector x £ R" in case when not many coordinates of x were affected 
by errors. For instance, “one can introduce errors of arbitrary large sizes and 
still recover the input vector exactly by solving a convenient linear program...”, 
see in [7]. To achieve such performance some special restriction on matrix H was 
placed, called Restricted Isometry Property (RIP), as follows 

(l-MIMh < \\Hx T \\ 2 < (l + S D )\\x\\ 2 , ( 8 ) 

for any vector a;GR":||:r||o<-D, where 0 < Sd < 1. The smallest possible 5d 
called the isometry constant. 

Then typical result in [7] (Th. 1.1 ) is of the following form 

“if S^t + 3 ( 54 t < 2 then the solution of linear programming problem is unique and 

equal to x” 

Let us note that the condition <5 3t + 3644 < 2 implies J 44 < 2/3 (of course, it 
implies that £44 < 1/2, but for us enough to have 644 < 1). Hence Hx T / 
0 for any nonzero x with wt(x) < 4 1, or in other words, an error-correcting 
code (over reals) corresponding to such parity-check matrix H has the minimal 
distance at least 4t + 1 and can correct 2 1 errors (instead of t). So we lost 
twice in error-correction capability but maybe linear programming provides more 
easier way for decoding ? In fact, NOT, since it is well known in coding theory 
that such problem can be solved rather easily (in complexity) over any infinite 
field by usage of the corresponding Reed-Solomon codes and known decoding 


algorithms. In case of real number or complex number fields one can use just 
an RS code with Fourier parity-check matrix, namely, hj tP = exp(2m —), where 
for complex numbers p £ {1, 2, n }, j = a, a + d, a + 2d,..., a + (r — l)d, 
and “reversible” RS-matrix H for real numbers, where p £ {1,2, ...,n}, j £ 
{-/, -/ + 1, • • ■, 0,1,..., /} and r = 2/ + 1. 

Fortunately, matrices with the RIP property allow to correct not only sparse 
errors but also additional errors with arbitrary support but relatively small (up 
to e) Euclidean norm. Again, the RIP property is good for linear programming 
decoding but is too strong in general. Namely, it is enough to have the following 
property 

A 2 t|M| 2 < \\Hx t \\ 2 , (9) 

for any vector x £ R" : ||x||o < 2 1, where A 2 1 > 0 and the largest such value 
we call extension constant. Indeed, then for any two solutions x and x of the 
Equation (1) we have that 


\\x - x\\ 2 <2X^6, (10) 

where ||e||a, ||e|I 2 < £■ Hence (10) shows that all solutions of the Equation (1) 
are “almost equal” if X 2 t is large enough. Let us note that for RS-matrices any 
r columns are linear independent and hence A 2 t > 0, but X 2 t tends to zero when 
n grows and code rate is fixed. To find better class of codes over the field of real 
(or complex) numbers is an open problem. 

5 Conclusion 

In this paper we extends technique, which was developed in uni for error correc¬ 
tion with errors in both the channel and syndrome, to the Compressed Sensing 
problem. We hope this approach will help to find limits of the unrestricted (i.e. 
without LP usage) compressed sensing. 
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